Integrating Neural Coding into Cognitive Models of Behaviour

The study of neural coding is complicated by the almost infinite number of possible codes: which of the constellation of codes is used by the brain? The approach adopted here is t o examine the validity of predictions about behaviour made from a simple model incorporating neural codes. The visual information acquisition hypothesis [1-3] predicts that neural activity should reveal a longer delay before the accumulated information reaches threshold as reaction times increase. An obvious neural correlate of changes in reaction time would be changes in neural response latency. Neural response latency increases with decreasing stimulus contrast [4-9] and is independent of neural response magnitude [4;9]. An alternative to changes in neural response latency underlying changes in behavioural reaction times is activity level. When the activation level of neural representations is reduced, the accumulation of evidence is slower, offering an alternative explanation of the need for mental transformation processes [10]. In V1, changes of image orientation produce changes in response magnitude with only little effect on response latency [4;6]. Image manipulations which change perspective view, object part visibility, orientation, size or direction of motion of stimuli in “higher” visual areas of the temporal lobe also have little effect on neural response latency [10-14] despite large changes in response magnitude [11;14]. Thus, there is considerable evidence that both changes in neural response latency and magnitude induced by changes in the input stimulus could underlie associated changes in behavioural reaction times. The use of the dual task paradigm allows distinctions to be made between the relationship of neural response latency and response magnitude coding with behavioural reaction times. Within the dual task paradigm, two stimuli are presented sequentially, each stimulus requiring a behavioural response. The initial processing of stimuli is assumed to be parallel (i.e. more than one stimulus can be processed at the same time). Subsequent serial processing is thought to prevent the decision based on one stimulus interfering, possibly catastrophically, with the decision based on the second stimulus. With short stimulus onset asynchronies (SOAs) in a dual task paradigm the serial “bottle-neck” apparatus is assumed to be processing the first stimulus when the second stimulus is presented. This leads to a delay before the serial processing of the second stimulus. With long SOAs there should be no delay before processing starts because the central “decision” processes have had time t o complete the first task. If changes in perceptual decision reaction times are driven by changes in the response latency of the underlying neural representation, then the recognition reaction time is predicted to be influenced by the image manipulation only at long SOA. [The assumed delay before monitoring the visual system at short SOA negates the increased response latency]. Alternatively, the change in predicted recognition reaction time with change in magnitude of response is independent of a delay in monitoring the signal. [The assumed delay before monitoring the visual system at short SOA is still influenced by response magnitude] Thus, in a dual task paradigm where the image manipulation changes neural response magnitude, the recognition reaction time is predicted to be influenced by the image manipulation regardless of the SOA. The time to threshold model based on neurophysiological data [10;15] was adapted to allow prediction of trial by trial variation in recognition times. A simple implementation of the time to threshold model allows the changes in the distribution of the RT to each stimulus condition to be made and compared to the observed distributions. The average response profile over time for the responses of single neurones to changes in the view of the head and body was used [11] in conjunction with the tuning profile of such neurones [16]. A population response consistent with observed population responses [10;11;16] was generated from 25 such neurones, with different neurones having different preferred views (i.e. stimulus orientations that elicited the maximal response). Statistical independence of the responses of each neurone was assumed, with the number of spikes elicited by a given stimulus on a trial to follow a mean~variance relationship described by a fano factor (variance/mean) of 1.5 [17-19]. The spike count matched model was used to simulate spike trains that have the same spike patterns within and between responses of different neurones (including synchronous spikes and synfire chains) as seen in neurophysiological data [18;19]. The population response was the summed spike trains across neurones. Time to threshold was calculated as the time when the accumulated spike count reached a given criterion. Estimates of the mean and standard deviation of time t o threshold to each stimulus were calculated from 10,000 simulated trials. For the human reaction times studies a variable auditory stimulus (1 – 4 pips) was followed by visual presentation of a single letter covering the fixation point, either 0 msecs or 1000 msecs after the end of the auditory stimulus (inter-trial interval varied uniformly 10001500 msec). The task was to decide (1) whether the number of pips was odd or even and (2) whether the letter was a normal or mirror image irrespective of the letter orientation. Responses were to be made as quickly and as accurately as possible. Each participant underwent a practice phase until achieving criterion level (70% correct). Blocks of 192 trials were presented with a participant controlled pause between blocks. The experimental phase was time-locked to a maximum of 20 minutes or 5 blocks of 192 trials. ANOVAs were performed using the log transformed RT (to remove the mean~sd relationship, [20]). Type III sums of squares and Greenhouse-Geiser corrections were used throughout. The results of the human reaction time study were in agreement with the pattern of results predicted from the time-to-threshold model. Analysis showed RT increasing systematically with the increasing rotation from upright. RT to low contrast images (mean±sem: 2249±169ms) was significantly greater than RT to high contrast images (1992±147ms). Consistent with the model, the effect of reducing stimulus contrast on RT was significantly smaller with a 0 ms delay between presentation of the auditory and visual stimuli (120 ms increase in RT with a 94% decrease in stimulus contrast) than with a 1000 ms delay (a 293 ms increase in RT). Also consistent with predictions from the time-to-threshold based model, the effect of stimulus contrast on RT was independent of rotation angle and the effect of stimulus rotation on RT was the same at both delays. The distribution of times for the model's activity to reach threshold showed a linear mean~standard deviation relationship. As response magnitude decreases, the mean and standard deviation of the time to threshold increase in a linear fashion. With a delay in the monitoring and accumulation of spike counts, there is a shift in the mean~SD relationship: the mean times to threshold increase, but the relationship between the mean and standard deviation remains constant (i.e. the slope is constant but shifted). These patterns of results were observed in the experimental data. The variability of RT seen with long stimulus asynchrony was significantly greater than the variability seen with short stimulus asynchrony. A significant decrease in the relative response variability was also observed when the stimulus contrast was increased. The model also predicted correctly that changes in the stimulus orientation showed a statistically constant variability of reaction time. In summary, the implications of neural coding of visual stimuli for cognitive processes were explored. The human cognitive experiment tested specific hypotheses generated from consideration of neurophysiological activity within the primate visual system. The results met the predictions about reaction times, including subtle changes in the mean~SD relationship of reaction times. These experiments show that neural coding may be combined with cognitive models to predict human behaviour. The model extends traditional cognitive models in that it is predictive (cognitive models typically involve a posteri descriptions of behaviour). Furthermore, the integration of neural coding ideas into cognitive models offers at least a partial mechanistic explanation of changes in the distributions of behavioural reaction times.